AI Model Inference (preview:2024-05-01)

2025/02/12 • 4 new methods

GetChatCompletions (new)
Description :  Gets chat completions for the provided chat messages. Completions support a wide variety of tasks and generate text that continues from or "completes" provided prompt data. The method makes a REST API call to the `/chat/completions` route on the given endpoint.
Reference :  Link ¶

⚼ Request

POST:  /chat/completions
{
api-version:
{
Description: The API version to use for this operation. ,
Required: True ,
}
string ,
extra-parameters:
{
Description: Controls what happens if extra parameters, undefined by the REST API, are passed in the JSON request payload. This sets the HTTP request header `extra-parameters`. ,
Required: False ,
}
string ,
body:
{
Description: The parameters of the chat completions request. ,
Required: True ,
}
{
messages:
{
Description: The collection of context messages associated with this chat completions request. Typical usage begins with a chat message for the System role that provides instructions for the behavior of the assistant, followed by alternating messages between the User and Assistant roles. ,
Required: True ,
}
[
{
role:
{
Description: The chat role associated with this message. ,
Enum:
{
system: The role that instructs or sets the behavior of the assistant. ,
developer: The role that provides instructions to the model prioritized ahead of user messages. ,
user: The role that provides input for chat completions. ,
assistant: The role that provides responses to system-instructed, user-prompted input. ,
tool: The role that represents extension tool activity within a chat completions operation. ,
}
,
Required: True ,
}
enum ,
}
,
]
,
frequency_penalty:
{
Description: A value that influences the probability of generated tokens appearing based on their cumulative frequency in generated text. Positive values will make tokens less likely to appear as their frequency increases and decrease the likelihood of the model repeating the same statements verbatim. Supported range is [-2, 2]. ,
Format: float ,
Required: False ,
}
number ,
stream:
{
Description: A value indicating whether chat completions should be streamed for this request. ,
Required: False ,
}
boolean ,
presence_penalty:
{
Description: A value that influences the probability of generated tokens appearing based on their existing presence in generated text. Positive values will make tokens less likely to appear when they already exist and increase the model's likelihood to output new topics. Supported range is [-2, 2]. ,
Format: float ,
Required: False ,
}
number ,
temperature:
{
Description: The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. Supported range is [0, 1]. ,
Format: float ,
Required: False ,
}
number ,
top_p:
{
Description: An alternative to sampling with temperature called nucleus sampling. This value causes the model to consider the results of tokens with the provided probability mass. As an example, a value of 0.15 will cause only the tokens comprising the top 15% of probability mass to be considered. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. Supported range is [0, 1]. ,
Format: float ,
Required: False ,
}
number ,
max_tokens:
{
Description: The maximum number of tokens to generate. ,
Format: int32 ,
Required: False ,
}
integer ,
response_format:
{
Description: An object specifying the format that the model must output. Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema. Setting to `{ "type": "json_object" }` enables JSON mode, which ensures the message the model generates is valid JSON. **Important:** when using JSON mode, you **must** also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length. ,
Required: False ,
}
{
type:
{
Description: The response format type to use for chat completions. ,
Required: True ,
}
string ,
}
,
stop:
{
Description: A collection of textual sequences that will end completions generation. ,
Required: False ,
}
[
string ,
]
,
tools:
{
Description: A list of tools the model may request to call. Currently, only functions are supported as a tool. The model may response with a function call request and provide the input arguments in JSON format for that function. ,
Required: False ,
}
[
{
type:
{
Description: The type of the tool. Currently, only `function` is supported. ,
Enum:
{
function: No description available. ,
}
,
Required: True ,
}
enum ,
function:
{
Description: The function definition details for the function tool. ,
Required: True ,
}
{
name:
{
Description: The name of the function to be called. ,
Required: True ,
}
string ,
description:
{
Description: A description of what the function does. The model will use this description when selecting the function and interpreting its parameters. ,
Required: False ,
}
string ,
parameters:
{
Description: The parameters the function accepts, described as a JSON Schema object. ,
Required: False ,
}
object ,
}
,
}
,
]
,
tool_choice:
{
Description: If specified, the model will configure which of the provided tools it can use for the chat completions response. ,
Required: False ,
}
string ,
seed:
{
Description: If specified, the system will make a best effort to sample deterministically such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed. ,
Format: int64 ,
Required: False ,
}
integer ,
model:
{
Description: ID of the specific AI model to use, if more than one model is available on the endpoint. ,
Required: False ,
}
string ,
modalities:
{
Description: The modalities that the model is allowed to use for the chat completions response. The default modality is `text`. Indicating an unsupported modality combination results in an 422 error. ,
Required: False ,
}
[
string ,
]
,
}
,
}

⚐ Response (200)

{
id:
{
Description: A unique identifier associated with this chat completions response. ,
Required: True ,
}
string ,
object:
{
Description: The response object type, which is always `chat.completion`. ,
Enum:
{
chat.completion: No description available. ,
}
,
Required: True ,
}
enum ,
created:
{
Description: The first timestamp associated with generation activity for this completions response, represented as seconds since the beginning of the Unix epoch of 00:00 on 1 Jan 1970. ,
Format: unixtime ,
Required: True ,
}
integer ,
model:
{
Description: The model used for the chat completion. ,
Required: True ,
}
string ,
choices:
{
Description: The collection of completions choices associated with this completions response. Generally, `n` choices are generated per provided prompt with a default value of 1. Token limits and other settings may limit the number of choices generated. ,
Required: True ,
}
[
{
index:
{
Description: The ordered index associated with this chat completions choice. ,
Format: int32 ,
Required: True ,
}
integer ,
finish_reason:
{
Description: The reason that this chat completions choice completed its generated. ,
Enum:
{
stop: Completions ended normally and reached its end of token generation. ,
length: Completions exhausted available token limits before generation could complete. ,
content_filter: Completions generated a response that was identified as potentially sensitive per content moderation policies. ,
tool_calls: Completion ended with the model calling a provided tool for output. ,
}
,
Required: True ,
}
enum ,
message:
{
Description: The chat message for a given chat completions prompt. ,
Required: True ,
}
{
role:
{
Description: The chat role associated with the message. ,
Enum:
{
system: The role that instructs or sets the behavior of the assistant. ,
developer: The role that provides instructions to the model prioritized ahead of user messages. ,
user: The role that provides input for chat completions. ,
assistant: The role that provides responses to system-instructed, user-prompted input. ,
tool: The role that represents extension tool activity within a chat completions operation. ,
}
,
Required: True ,
}
enum ,
content:
{
Description: The content of the message. ,
Required: True ,
}
string ,
tool_calls:
{
Description: The tool calls that must be resolved and have their outputs appended to subsequent input messages for the chat completions request to resolve as configured. ,
Required: False ,
}
[
{
id:
{
Description: The ID of the tool call. ,
Required: True ,
}
string ,
type:
{
Description: The type of tool call. Currently, only `function` is supported. ,
Enum:
{
function: No description available. ,
}
,
Required: True ,
}
enum ,
function:
{
Description: The details of the function call requested by the AI model. ,
Required: True ,
}
{
name:
{
Description: The name of the function to call. ,
Required: True ,
}
string ,
arguments:
{
Description: The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function. ,
Required: True ,
}
string ,
}
,
}
,
]
,
audio:
{
Description: The audio generated by the model as a response to the messages if the model is configured to generate audio. ,
Required: False ,
}
{
id:
{
Description: Unique identifier for the audio response. This value can be used in chat history messages instead of passing the full audio object. ,
Required: True ,
}
string ,
expires_at:
{
Description: The Unix timestamp (in seconds) at which the audio piece expires and can't be any longer referenced by its ID in multi-turn conversations. ,
Format: unixtime ,
Required: True ,
}
integer ,
data:
{
Description: Base64 encoded audio data ,
Required: True ,
}
string ,
format:
{
Description: The format of the audio content. If format is not provided, it will match the format used in the input audio request. ,
Enum:
{
wav: Specifies audio in WAV format. ,
mp3: Specifies audio in MP3 format. ,
}
,
Required: False ,
}
enum ,
transcript:
{
Description: The transcript of the audio file. ,
Required: True ,
}
string ,
}
,
}
,
}
,
]
,
usage:
{
Description: Usage information for tokens processed and generated as part of this completions operation. ,
Required: True ,
}
{
completion_tokens:
{
Description: The number of tokens generated across all completions emissions. ,
Format: int32 ,
Required: True ,
}
integer ,
prompt_tokens:
{
Description: The number of tokens in the provided prompts for the completions request. ,
Format: int32 ,
Required: True ,
}
integer ,
total_tokens:
{
Description: The total number of tokens processed for the completions request and response. ,
Format: int32 ,
Required: True ,
}
integer ,
completion_tokens_details:
{
Description: Breakdown of tokens used in a completion. ,
Required: False ,
}
{
audio_tokens:
{
Description: The number of tokens corresponding to audio input. ,
Format: int32 ,
Required: True ,
}
integer ,
total_tokens:
{
Description: The total number of tokens processed for the completions request and response. ,
Format: int32 ,
Required: True ,
}
integer ,
}
,
prompt_tokens_details:
{
Description: Breakdown of tokens used in the prompt/chat history. ,
Required: False ,
}
{
audio_tokens:
{
Description: The number of tokens corresponding to audio input. ,
Format: int32 ,
Required: True ,
}
integer ,
cached_tokens:
{
Description: The total number of tokens cached. ,
Format: int32 ,
Required: True ,
}
integer ,
}
,
}
,
}

⚐ Response (default)

{
$headers:
{
x-ms-error-code:
{
Description: String error code indicating what went wrong. ,
}
string ,
}
,
$schema:
{
Description: A response containing error details. ,
}
{
error:
{
Description: The error object. ,
Required: True ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: True ,
}
string ,
message:
{
Description: A human-readable representation of the error. ,
Required: True ,
}
string ,
target:
{
Description: The target of the error. ,
Required: False ,
}
string ,
details:
{
Description: An array of details about specific errors that led to this reported error. ,
Required: False ,
}
[
string ,
]
,
innererror:
{
Description: An object containing more specific information than the current object about the error. ,
Required: False ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: False ,
}
string ,
innererror:
{
Required: False ,
}
string ,
}
,
}
,
}
,
}
GetEmbeddings (new)
Description :  Return the embedding vectors for given text prompts. The method makes a REST API call to the `/embeddings` route on the given endpoint.
Reference :  Link ¶

⚼ Request

POST:  /embeddings
{
api-version:
{
Description: The API version to use for this operation. ,
Required: True ,
}
string ,
extra-parameters:
{
Description: Controls what happens if extra parameters, undefined by the REST API, are passed in the JSON request payload. This sets the HTTP request header `extra-parameters`. ,
Required: False ,
}
string ,
body:
{
Description: The parameters of the embeddings request. ,
Required: True ,
}
{
input:
{
Description: Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. ,
Required: True ,
}
[
string ,
]
,
dimensions:
{
Description: Optional. The number of dimensions the resulting output embeddings should have. Passing null causes the model to use its default value. Returns a 422 error if the model doesn't support the value or parameter. ,
Format: int32 ,
Required: False ,
}
integer ,
encoding_format:
{
Description: Optional. The desired format for the returned embeddings. ,
Enum:
{
base64: Get back binary representation of the embeddings encoded as Base64 string. OpenAI Python library retrieves embeddings from the API as encoded binary data, rather than using intermediate decimal representations as is usually done. ,
binary: Get back signed binary embeddings ,
float: Get back full precision embeddings ,
int8: Get back signed int8 embeddings ,
ubinary: Get back unsigned binary embeddings ,
uint8: Get back unsigned int8 embeddings ,
}
,
Required: False ,
}
enum ,
input_type:
{
Description: Optional. The type of the input. Returns a 422 error if the model doesn't support the value or parameter. ,
Enum:
{
text: Indicates the input is a general text input. ,
query: Indicates the input represents a search query to find the most relevant documents in your vector database. ,
document: Indicates the input represents a document that is stored in a vector database. ,
}
,
Required: False ,
}
enum ,
model:
{
Description: ID of the specific AI model to use, if more than one model is available on the endpoint. ,
Required: False ,
}
string ,
}
,
}

⚐ Response (200)

{
id:
{
Description: Unique identifier for the embeddings result. ,
Required: True ,
}
string ,
data:
{
Description: Embedding values for the prompts submitted in the request. ,
Required: True ,
}
[
{
embedding:
{
Description: List of embedding values for the input prompt. These represent a measurement of the vector-based relatedness of the provided input. Or a base64 encoded string of the embedding vector. ,
Required: True ,
}
[
number ,
]
,
index:
{
Description: Index of the prompt to which the EmbeddingItem corresponds. ,
Format: int32 ,
Required: True ,
}
integer ,
object:
{
Description: The object type of this embeddings item. Will always be `embedding`. ,
Enum:
{
embedding: No description available. ,
}
,
Required: True ,
}
enum ,
}
,
]
,
usage:
{
Description: Usage counts for tokens input using the embeddings API. ,
Required: True ,
}
{
prompt_tokens:
{
Description: Number of tokens in the request. ,
Format: int32 ,
Required: True ,
}
integer ,
total_tokens:
{
Description: Total number of tokens transacted in this request/response. Should equal the number of tokens in the request. ,
Format: int32 ,
Required: True ,
}
integer ,
}
,
object:
{
Description: The object type of the embeddings result. Will always be `list`. ,
Enum:
{
list: No description available. ,
}
,
Required: True ,
}
enum ,
model:
{
Description: The model ID used to generate this result. ,
Required: True ,
}
string ,
}

⚐ Response (default)

{
$headers:
{
x-ms-error-code:
{
Description: String error code indicating what went wrong. ,
}
string ,
}
,
$schema:
{
Description: A response containing error details. ,
}
{
error:
{
Description: The error object. ,
Required: True ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: True ,
}
string ,
message:
{
Description: A human-readable representation of the error. ,
Required: True ,
}
string ,
target:
{
Description: The target of the error. ,
Required: False ,
}
string ,
details:
{
Description: An array of details about specific errors that led to this reported error. ,
Required: False ,
}
[
string ,
]
,
innererror:
{
Description: An object containing more specific information than the current object about the error. ,
Required: False ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: False ,
}
string ,
innererror:
{
Required: False ,
}
string ,
}
,
}
,
}
,
}
GetImageEmbeddings (new)
Description :  Return the embedding vectors for given images. The method makes a REST API call to the `/images/embeddings` route on the given endpoint.
Reference :  Link ¶

⚼ Request

POST:  /images/embeddings
{
api-version:
{
Description: The API version to use for this operation. ,
Required: True ,
}
string ,
extra-parameters:
{
Description: Controls what happens if extra parameters, undefined by the REST API, are passed in the JSON request payload. This sets the HTTP request header `extra-parameters`. ,
Required: False ,
}
string ,
body:
{
Description: The parameters of the image embeddings request. ,
Required: True ,
}
{
input:
{
Description: Input image to embed. To embed multiple inputs in a single request, pass an array. The input must not exceed the max input tokens for the model. ,
Required: True ,
}
[
{
image:
{
Description: The input image encoded in base64 string as a data URL. Example: `data:image/{format};base64,{data}`. ,
Required: True ,
}
string ,
text:
{
Description: Optional. The text input to feed into the model (like DINO, CLIP). Returns a 422 error if the model doesn't support the value or parameter. ,
Required: False ,
}
string ,
}
,
]
,
dimensions:
{
Description: Optional. The number of dimensions the resulting output embeddings should have. Passing null causes the model to use its default value. Returns a 422 error if the model doesn't support the value or parameter. ,
Format: int32 ,
Required: False ,
}
integer ,
encoding_format:
{
Description: Optional. The number of dimensions the resulting output embeddings should have. Passing null causes the model to use its default value. Returns a 422 error if the model doesn't support the value or parameter. ,
Enum:
{
base64: Get back binary representation of the embeddings encoded as Base64 string. OpenAI Python library retrieves embeddings from the API as encoded binary data, rather than using intermediate decimal representations as is usually done. ,
binary: Get back signed binary embeddings ,
float: Get back full precision embeddings ,
int8: Get back signed int8 embeddings ,
ubinary: Get back unsigned binary embeddings ,
uint8: Get back unsigned int8 embeddings ,
}
,
Required: False ,
}
enum ,
input_type:
{
Description: Optional. The type of the input. Returns a 422 error if the model doesn't support the value or parameter. ,
Enum:
{
text: Indicates the input is a general text input. ,
query: Indicates the input represents a search query to find the most relevant documents in your vector database. ,
document: Indicates the input represents a document that is stored in a vector database. ,
}
,
Required: False ,
}
enum ,
model:
{
Description: ID of the specific AI model to use, if more than one model is available on the endpoint. ,
Required: False ,
}
string ,
}
,
}

⚐ Response (200)

{
id:
{
Description: Unique identifier for the embeddings result. ,
Required: True ,
}
string ,
data:
{
Description: Embedding values for the prompts submitted in the request. ,
Required: True ,
}
[
{
embedding:
{
Description: List of embedding values for the input prompt. These represent a measurement of the vector-based relatedness of the provided input. Or a base64 encoded string of the embedding vector. ,
Required: True ,
}
[
number ,
]
,
index:
{
Description: Index of the prompt to which the EmbeddingItem corresponds. ,
Format: int32 ,
Required: True ,
}
integer ,
object:
{
Description: The object type of this embeddings item. Will always be `embedding`. ,
Enum:
{
embedding: No description available. ,
}
,
Required: True ,
}
enum ,
}
,
]
,
usage:
{
Description: Usage counts for tokens input using the embeddings API. ,
Required: True ,
}
{
prompt_tokens:
{
Description: Number of tokens in the request. ,
Format: int32 ,
Required: True ,
}
integer ,
total_tokens:
{
Description: Total number of tokens transacted in this request/response. Should equal the number of tokens in the request. ,
Format: int32 ,
Required: True ,
}
integer ,
}
,
object:
{
Description: The object type of the embeddings result. Will always be `list`. ,
Enum:
{
list: No description available. ,
}
,
Required: True ,
}
enum ,
model:
{
Description: The model ID used to generate this result. ,
Required: True ,
}
string ,
}

⚐ Response (default)

{
$headers:
{
x-ms-error-code:
{
Description: String error code indicating what went wrong. ,
}
string ,
}
,
$schema:
{
Description: A response containing error details. ,
}
{
error:
{
Description: The error object. ,
Required: True ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: True ,
}
string ,
message:
{
Description: A human-readable representation of the error. ,
Required: True ,
}
string ,
target:
{
Description: The target of the error. ,
Required: False ,
}
string ,
details:
{
Description: An array of details about specific errors that led to this reported error. ,
Required: False ,
}
[
string ,
]
,
innererror:
{
Description: An object containing more specific information than the current object about the error. ,
Required: False ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: False ,
}
string ,
innererror:
{
Required: False ,
}
string ,
}
,
}
,
}
,
}
GetModelInfo (new)
Description :  Returns information about the AI model. The method makes a REST API call to the `/info` route on the given endpoint. This method will only work when using Serverless API or Managed Compute endpoint. It will not work for GitHub Models endpoint or Azure OpenAI endpoint.
Reference :  Link ¶

⚼ Request

GET:  /info
{
api-version:
{
Description: The API version to use for this operation. ,
Required: True ,
}
string ,
}

⚐ Response (200)

{
model_name:
{
Description: The name of the AI model. For example: `Phi21` ,
Required: True ,
}
string ,
model_type:
{
Description: The type of the AI model. A Unique identifier for the profile. ,
Enum:
{
embeddings: A model capable of generating embeddings from a text ,
chat-completion: A model capable of taking chat-formatted messages and generate responses ,
}
,
Required: True ,
}
enum ,
model_provider_name:
{
Description: The model provider name. For example: `Microsoft Research` ,
Required: True ,
}
string ,
}

⚐ Response (default)

{
$headers:
{
x-ms-error-code:
{
Description: String error code indicating what went wrong. ,
}
string ,
}
,
$schema:
{
Description: A response containing error details. ,
}
{
error:
{
Description: The error object. ,
Required: True ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: True ,
}
string ,
message:
{
Description: A human-readable representation of the error. ,
Required: True ,
}
string ,
target:
{
Description: The target of the error. ,
Required: False ,
}
string ,
details:
{
Description: An array of details about specific errors that led to this reported error. ,
Required: False ,
}
[
string ,
]
,
innererror:
{
Description: An object containing more specific information than the current object about the error. ,
Required: False ,
}
{
code:
{
Description: One of a server-defined set of error codes. ,
Required: False ,
}
string ,
innererror:
{
Required: False ,
}
string ,
}
,
}
,
}
,
}